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ABSTRACT 

The purpose of this paper is to discuss one instructional design that leverages problem-based learning and game 
structures as a means of developing innovative higher education courses for students as responsive, lived experiences. 
This paper reviews a curricular redesign that stemmed from the evaluation of an introductory course in computer 
applications that had high drop, failure, and withdrawal [D/F/W] rates. Interviews with students and faculty in this course 
revealed that students were not engaged with, motivated by, or satisfied with the instructional methods, which were 
often frustrating and difficult to navigate. Using data collected from students and faculty, we describe the full redesign of 
the course, which included ill-structured problems for students to solve, multiple forms of learning assessment, and a 
contextual framing stemming from a digital, alternate reality game design. When comparing the new design to the 
original, the first iteration research indicated decreased failure rates, increased achievement on standardized 
assessments, and a range of individual student experiences from high praise of the design to some disappointment. 
Keywords: Undergraduate, Computer Literacy, Problem-based Learning, Course Redesign. 


INTRODUCTION 

"Many students graduate having accumulated whatever 
number of courses is required, but still lacking a coherent 
body of knowledge or any inkling as to how one sort of 
information might relate to others. (A)ll too often they 
graduate without knowing how to think logically, write 
clearly, or speak coherently...(t)he university has given 
them too little that will be of real value beyond a credential 
that will help them get their first jobs." The Boyer Commission 
on Educating Undergraduates (1995). 

It has been more than a decade since the Boyer 
Commission (1995) suggested that post-secondary 
learning in the United States is failing our undergraduate 
students by not preparing them for the world of work that 
follows their academic studies. Further, it has also been 
suggested that the teaching methods used by course 
instructors may not have kept pace with the needs of an 
advancing society or the demands of the emerging world 
economy (Carini & Kuh, 2003; Fritschner, 2000; Kuh, 2001). 
An important area of weakness in the preparation of 


undergraduate students has been that their critical thinking 
abilities are not adequately developed, resulting in poor 
application of appropriate strategies to solve problems in 
different contexts (Ironside, 2003; Kallenbach & Viens, 
2004). 

With this weakness in mind, one particular effort to enhance 
the quality of the undergraduate experience in large, 
introductory courses at a mid-sized Texas university was 
developed in order to address troubling findings revealed 
by institutional data. By the spring of 2006, the university had 
identified several hundred sections of large enrollment 
undergraduate courses from which a significant number of 
students dropped within the first seven days, withdrew prior 
to the mid-semester deadline, or earned a grade of D or F. 
In most instances, these classes had enrollments larger 
than 150 in traditional lecture classes or greater than 30 in 
those that required computer laboratories. 

To address this problem, institutional leadership initiated a 
program for redesigning courses delivered through large 
group instruction (LGI). Course redesigns were required to 
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include technology and teaching methods, specifically 
small group activities, that made large classes seem 
smaller or better engaged students in meaningful 
interactions with peers and instructors than the traditional 
lecture format, The goals of each course redesign was to 
improve student satisfaction, thereby reducing the 
Drop/Failure/Withdrawal rate, factors often adversely 
affected by LGI (Peterson & Miller, 2004). Enhancing critical 
thinking or problem solving skills was also a primary goal of 
these redesign efforts. 

One course that underwent redesign was CECS 1100: 
Introduction to Computer Applications. This laboratory 
class (<33 students) was taught through the College of 
Education. The course introduced students to the history of 
the Internet, Microsoft Office Suite, and basic web tools. In 
addition to the university's goals, our purpose with the 
course redesign was to better prepare undergraduates for 
their future work beyond the classroom as professionals in 
the workplace, using technology tools to communicate 
effectively with peers and future employers. 

Theoretical foundations 

The course redesign leveraged elements of Problem- 
based Learning (PBL) (Albanese & Mitchell, 1993) and 
constructivist learning environments (CLEs) (Jonassen, 
1999; Jonassen & Hernandez-Serrano, 2002) because of 
their reported impact on critical thinking and problem 
solving skills (Meyerson & Adams, 2003; Sage, 2002; Willis, 
2002; Yip & Gafarian, 2002). This choice stemmed from the 
fact that the course emphasized skill acquisition and 
neither critical thinking nor problem solving were addressed 
in the previous design, which focused instead on 
acquisition of surface level skills and memorization. 

Research has shown that PBL encourages learners to hone 
a variety of thinking skills: 

• analyze and synthesize data; 

• develop hypotheses; 

• apply deductive reasoning to a problem situation; 

• draw conclusions after analysis, synthesis and 
evaluation of new information; 

• synthesize strategies/solutions; and 

• monitor and evaluate own thinking process" (Tiwari & 


Lai, 2002, p. 2). 

In order to support this further, other designers have 
combined a PBL approach with proven critical thinking 
strategies that have been developed outside of PBL to 
enhance the existing strengths of the approach while 
providing additional scaffolding for learners (DiPasquale, 
Mason, & Kolkhorst, 2003; Elder & Paul, 2002; Everett & 
Zinser, 1998; Keller, 2002; Willis, 2002) such as the use of 
stories and narrative to contextualize experience and the 
inclusion small group tasks. 

Thus, the framework for the redesigned course included 
elements from Savery and Duffy's (1995) PBL design 
principles, particularly an authentic, interactive learning 
environment, cognitive conflict, and opportunities for 
social negotiation, because according to these authors, 
"(i) Understanding is in our interactions with the environment 
... (ii) Cognitive conflict or puzzlement is the stimulus for 
learning and determines the organization and nature of 
what is learned ... and (iii) Knowledge evolves through 
social negotiation and through the evaluation of the 
viability of individual understandings" (p. 1-2). The design 
team relied on Jonassen's (1999) criteria for creating the ill- 
structured problems, which shape the core of PBL 
environments. These criteria include unstated goals and 
constraints, uncertainty about which concepts and 
principles are necessary for the solution, as well as multiple 
criteria for evaluating problem solutions (Jonassen, 1999). 

Interactive learning games 

Not only has PBL been correlated with a propensity for 
enhancing critical thinking skills (Tiwari & Lai, 2002) and 
improving post-secondary learning experiences by 
providing authentic tasks (Bonk, Kirkley, Hara, & Denned, 
2001), it also compels students to engage in story-driven 
learning tasks much like those found in video games. PBL 
makes sense as a basis for developing a form of game 
scenario at the post-secondary level (Albanese & Mitchell, 
1993; DiPasquale, et al., 2003; Elder & Paul, 2002; Keller, 
2002; Kolodner, 2002; Willis, 2002; Zembal-Saul, 
Blumenfeld, & Krajcik, 2000) because this instructional 
model requires the presence of rules governing student 
and instructor interaction and roles, artificial conflict in the 
form of ill-structured problems, and "win scenarios" in which 
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students have successfully or sufficiently developed a 
defensible solution to the problem or conflict. 
Compounded by the fact that post-secondary learners 
tend to be motivated by internal self-esteem, recognition, 
need for a better quality of life, and self-actualization rather 
than extrinsic rewards (Rachal, 2002; Shank, Winchell, & 
Myers, 2001; Terehoff, 2002), the learning activities were 
packaged in such a way that they mirrored real-world tasks 
contextualized by the narrative overlay common in 
computer and video games that has been found to 
motivate learners, particularly those that require additional 
feedback, peer support, and motivations that they may not 
find in more traditional classrooms. 

However, given the complications and expense of 
designing immersive game worlds that include both the 
narrative plot and the requisite scaffolds to facilitate 
learning (Warren & Jones, 2008), alternative media that 
leverage these elements may be used (Jones & Warren, 
2008). One such alternative is to embed game activities 
that reveal enabling information and resources in a variety 
of media, distributed across the Internet rather than a fully 
integrated, stand-alone product. This approach maximizes 
resources, such as MySpace, web logs, podcasts, YouTube, 
and the three-dimensional digital environment of Second 
Life that students use as part of their daily lives (Dondlinger & 
Warren, 2008; Warren & Dondlinger, 2009). As such, it 
creates an open system of resource distribution that more 
authentically mirrors the context to which learners will 
transfer the skills and knowledge gained in the learning 
activities. 

Alternate Reality Games (AltRGs) 

A fairly new genre of game, the Alternate Reality Game 
(AltRG) does just that: distributes game challenges, tasks, 
and rewards across a variety of media both digital and real. 
As described by the International Game Developers 
Association (Martin & Chatfield, 2006), "Alternate Reality 
Games take the substance of everyday life and weave it 
into narratives that layer additional meaning, depth, and 
interaction upon the real world" (p. 6). CNET staff writer, John 
Borland (2005), depicts them as "an obsession-inspiring 
genre that blends real-life treasure hunting, interactive 
storytelling, video games and online community" (para. 4). 


Although the AltRG as a game genre has emerged quite 
recently, web sites like unfiction.com and ARGNet have 
links to several past and current AltRGs, such as 
cathysbook.com and iamtryingtobelieve.com. Some 
AltRGs have served a marketing function, such as 
ilovebees.com that supported the release of Microsoft's 
Halo 2 video game. Others have an educational focus; 
Hexagon Challenge and Never Rest Game are billed as 
instructional AltRGs and claim to "address decision-making 
skills, after-action report generation, and adaptation to 
performance" (Bogost, 2007). Yet others, while not explicitly 
educational, deal with social, economic, or environmental 
justice and aim "to change the way people think, and feel, 
and live" (Strickland, 2007). Jane McGonigal, who 
masterminds AltRGs for their capacity to construct 
"collective intelligence," maintains that the purpose of her 
2007 ARG World Without Oil (Ernst, 2007) was to "play our 
way to a set of ideas about how to manage that crisis [a 
dramatic decrease in oil availability]", (cited in Strickland, 
2007, p. 1) McGonigal observed that players not only 
generated strategies for coping with a peak oil crisis, they 
also changed their real world behavior: planting trees or 
converting their cars to run on biodiesel (Strickland, 2007). 
Thus, the simulated problem presented through the ARG 
yielded practical solutions and prompted real world 
applications of the knowledge constructed in the 
simulated play space. 

Analysis of Existing Course Materials 

The original course was designed to teach the basics of 
computer parts, Internet use, online security, and the use of 
Microsoft Office as implemented in Adobe Flash-based 
Computer Assisted Instruction (CAI) program called SAMS 
2003 Computer Literacy™ ‘Thomson Course Technology, 
2007). Analysis of the course materials and notes from 
interviews with students in the existing course revealed 
several issues that led to dissatisfaction with the existing 
curriculum and instructional methods used in the course. 
The design team identified the following problems: 

• Functions in one program were not linked to others. 
Understanding how the programs can be used in a 
complementary fashion is an important objective that 
wasn't addressed in the existing curriculum. Students 
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found learning the same skills in four different programs 
boring and repetitive, since many students were 
already familiar with these basic actions from previous 
computer applications courses in high school. 

• Computer-based assessments and instruction were 
too rigid. There are commonly three to five ways to 
perform an action in a program; however, the 
program often recognized only one. In some 
instances, the practice exercise asked a student to 
complete a task in one manner, while the assessment 
compelled another. 

• Applications of knowledge were decontextualized and 
unrealistic. Based on students' current life experiences, 
the applications of learning expected in both the 
training practice and exams don't fit well with an 
undergraduate's life experiences. Students saw little 
relevance between course content and their future 
work. They also found the constant drill and practice 
tedious. 

• Computer-based instruction provided weak 
feedback. The system provided weak feedback during 
both training and often none during exams. The 
amount of feedback did not increase or decrease 
dependent on how many correct or incorrect answers 
students provided for a specific objective. 

Based on this analysis, the design team determined that 

the following measures be faken to address the underlying 

problems: 

• The number of discrete learning objectives should be 
revised from 750 to 150. Given the length of the course, 
the sheer number of objectives was overwhelming to 
students, and should be collapsed to eliminate 
redundancies from unit to unit. For example, the 
objectives "The learner will be able to open an MS Word 
document" and "The learner will be able to open an 
Excel spreadsheet" should be consolidated to read, 
"The learner will be able to open documents within MS 
Office." 

• Requirements for the course should be stable across 
sections and semesters, but revised yearly to 
incorporate innovations in the field. With rapid changes 
in information technology and differing needs of 


students, course requirements should be updated 
regularly. An examination of state technology 
requirements for K-12 learning should take place yearly 
to ensure that the course does not simply re-teaching 
the same concepts students learned in high school. 

• The course should be centered on larger learning 
projects and problem solving using the software, not 
around disembodied learning tasks. The nature of the 
computer applications introduced in the course 
readily lends them to a project-based or contextual 
learning approach. To better engage students with 
these tools in the manner for which they're intended, 
the learning tasks should leverage them as a means to 
solve an ill-structured problem, design a project, or 
effectively communicate ideas to others. 
Development of appropriate, rubric-based 
assessments rather than multiple-choice tests is also 
warranted. 

Instructional Design 

The university's retention goals, the research literature, and 
analysis of the existing course supported a redesign using 
problem-based learning methods. Furthermore, the use of 
story-like scenarios typical of problem-based learning (PBL) 
(Savery & Duffy, 1995) is a prominent element in digital 
games, and media products known to engage players for 
hours on end. However, given the challenge and cost of 
designing an immersive game world, alternative media 
that leverages that both narrative plot and the requisite 
learning scaffolds to facilitate learning is necessary. One 
such alternative is to embed game activities and resources 
in a variety of media, distributed across the Internet using 
Alternate Reality Game (AltRG) structures (Martin & 
Chatfield, 2006; Terdiman, 2008), rather than a fully 
integrated, stand-alone product. This approach maximizes 
resources, such as MySpace, generic web logs. Podcasts, 
YouTube, and the three-dimensional digital environment of 
Linden Labs' Second Life. 

Teamwork and learning game resource distribution 

Within the course curriculum, students were expected to 
work in small teams of 3-4 students to solve problems 
posed by fictional clients similar to those they may 
encounter in a video game that last for two weeks; it is 
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during this time they explore the resources provided to 
them or uncover more as they play the game. This helps 
create an open system of resource distribution that 
authentically mirrors the contexts to which learners will 
transfer the skills and knowledge once they are done with 
college and working in the real world. This concept also 
allows designers to exploit many free online resources while 
merging them with PBL methods to give learners a situated, 
coherent narrative to contextualize their learning 
experience while concurrently providing cognitive 
scaffolds from which to retrieve knowledge and skills 
necessary to their future work and learning. In order for 
students to cognitively transition from an acquisition model 
to knowledge construction requires curricular and 
instructional innovation. 

Hybrid course format and technology tools 

Students in this class met face-to-face four to ten times 
during the semester, depending on instructor and student 
preference, to practice the basic skills needed to 
complete coursework. The remainder of the time, the class 
met in small groups online in the Second Life digital 
environment, which is free to download and enter. Visual 
imagery and audio provide information, tasks to complete, 
and a larger narrative structure within which students may 
situate their learning. Rather than listening to lectures and 
taking large multiple-choice tests, students will hone their 
technology skills by solving ill-structured problems that they 
encounter in the 3-D environment. They worked in small 
groups, using productivity tools to develop products that 
solve posed problems, and take part in support one 
another using a complimentary courseware tool called 
Moodle. The goal of this instruction was to provide students 
with a general set of skills that will allow them to use any 
word processor, spreadsheet program, or presentation tool 
and adapt to new versions readily. Problems were 
contextualized within a larger narrative structure that takes 
students along through a linear story within which their 
understanding was be situated. Game structures such as 
an overarching conflict, objectives to complete, clues to 
seek out and interpret, and feedback from the 3-D system 
were expected to increase student engagement with the 
learning of basic computer skills and knowledge. 


PBL and AltRG design 

Both problem-based learning (PBL) and alternate reality 
game structures were used to redesign the computer 
applications course. Rather than listen to lectures, 
complete practice exercises, and take frequent multiple- 
choice tests, students hone their technology skills by solving 
a series of ill-structured problems posed by fictional clients 
using the very tools they are expected to learn. Students 
work on each task or problem in small groups of two or 
three, using a variety of productivity and communication 
tools. The redesign made use of a hybrid or blended 
learning format. Face-to-face class time was dedicated to 
delivering instruction and to facilitate group problem 
solving. Online resources, support, and collaboration tools 
were also provided through the free courseware platform, 
Moodle. However, students were encouraged to make use 
of whatever productivity and communication tools best fit 
the dynamic of their groups. Emphasis was placed on 
communicating with peers, in class and online, to develop 
viable and deliverable solutions, rather than enforcing 
conformity to a specific version of a designated proprietary 
software program. 

The goal of this instruction was to provide students with a 
general set of skills that would allow them to use any word 
processor, spreadsheet program, or presentation tool and 
adapt to new versions readily. It also addressed some of the 
issues cited previously that frequently accompany online 
learning and digital collaboration, compelling students to 
negotiate solutions to issues of accessibility, software 
compatibility, and file management in their own teams. For 
example, if one team member could not afford the latest 
version of Office, the team might use Sun's Open Office or 
Google Docs. These broadened objectives were expected 
to better prepare students for their future world of work. 

Alternate Reality Game (AltRG) course structure 

The Door AltRG was designed with a two-tiered narrative 
structure that framed course activities and provided the 
context for problem solving. The first tier of this narrative 
engaged students with fictional clients who "hired" student 
teams to complete authentic tasks — a problem-based 
narrative approach. The second tier engaged students in 
game structures that included puzzles, codes, and ciphers 
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that must be solved, retrieved or used correctly in order to 
gain access to materials, information, and resources that 
provide additional scaffolding and narrative support to the 
first tier learning tasks, In essence, each of the clients and 
characters in the six, problem-based learning scenarios 
had alternate personas, hidden beneath their client 
identities, and all of them were embroiled in an underlying 
conflict with each other as well as the unsuspecting student 
players. Within the top-level story of The Door, students are 
asked by "clients" to solve complex, ill-structured problems 
that require them to use all the major components of 
Microsoft Office, The problems students faced ranged in 
complexity. In one instance, students had to provide 
directions to an inept old coach and gym teacher for how 
to construct a properly functioning grade book 
spreadsheet in order to allow him to keep his job at a 
middle school. In another instance, students develop an 
improved web site for a local nightclub that included 
appropriate use of basic color theory and space usage. 

At the same time, clues appear indicating that a software 
program called the Autumnal Equinox Firewall has 
disappeared which may have dire consequences for both 
the students and the world. Through these clues, the 
second tier of the story is revealed. The Puppet Master 
character of the game, Hester, offers students rewards for 
locating relevant game information. Further, she notes that 
additional resources to improve their problem solutions will 
be revealed if they obtain these rewards, such as a video 
that students can locate in YouTube (link here) if they put 
together a web address correctly. 

In this way, the Puppet Master, played by the instructor, 
provides soft scaffolding and additional resources for 
students who may be struggling either to solve the ill- 
structured problems or locate game resources. Game 
characters also act as gatekeepers, judging the quality of 
student solutions and preventing them from moving to the 
next problem until the last has been adequately 
addressed. As students move through the story at both 
levels, clues and minor puzzles are revealed. If students are 
successful at piecing this information together, they may 
discover that the clients are intended to be the ancient 
Greek gods seeking to reclaim followers and power by 


harnessing the power of the Internet, a power these 
students seek to understand. This included web sites we 
created, Ning, Moodle, and e-mail all linked together as 
shown in Figure 1. 

Visual imagery and audio were used to provide objective 
information about the ill-structured learning tasks and to 
spur communications among students, instructors, and 
game characters. Further, the two-tiered narrative framing 
provided the means for students to situate their learning in a 
more meaningful and engaging context (Warren, Stein, 
Dondlinger, & Barab, 2009), It also encouraged students to 
interrogate the inconsistencies between the two plotlines 
and was leveraged to challenge students to rethink their 
surface-level understandings of what was presented to 
them by the game. 

Student learning outcomes and assessment. As noted 
earlier, student learning outcomes (SLO) were reduced 
from more than 750 separate outcomes/learning 
objectives to 150, mainly by eliminating those that were 
repetitive from one MS Office program to the next. The 
objectives that emerged may be found here. These 
outcomes were only those specifically related to surface- 
level learning related to either memorization of facts or 
application of skills based on rote memorization. 

Rationale for the study 

Existing literature indicates that students are motivated by 
video games to engage in tasks that they are traditionally 



Figure 1. Distributed resources, clues, and communication 
tools for the ALFRG. 


i-manager’s Journal of Educational Technology, Vol. 7 • No. T • April - June 2010 


45 




















RESEARCH PAPERS 


resistant to in school. Few studies have engaged 
undergraduates in game-based curriculum tied to ill- 
structured problems requiring high levels of group 
knowledge construction and problem-solving and most 
that have indicate no significant gains on achievement 
tests when compared with traditional, direct instruction 
curriculum. The authors expected that merging game and 
problem-based learning elements into an immersive 
experience should result in both not only learning gains on 
an objective test, but also lead to improvements in student 
experience with the course that in turn reduce drop, failure, 
and withdrawal rates when compared with others that do 
not employ this pedagogical and curricular approach. 

Purpose of the study 

As with any new instructional methodology, it is as important 
to discover the specific answers as to why it was more 
effective than another method as it is to learn that it is 
effective at modifying behavior, improving achievement, 
or engaging learners in reflection associated with 
improvements in critical thinking. Therefore, the purpose of 
this study was to discover whether the game and PBL- 
based CECS 1100 undergraduate curriculum could 
improve student achievement, drop, failure, and 
withdrawal (D/F/W) rates, and general satisfaction with the 
CECS 1100 course when compared with a course that 
continued to use the old computer-based instruction 
curriculum, 

Research Methods 
Participants and site 

Eighty-nine students at a large southwestern university who 
enrolled in the introductory computer literacy course 
participated in the study. These students came from a wide 
range of colleges and majors ranging from Education and 
Business to Arts and Sciences, Sixty-one of these students 
were female and 28 male. The study took place in a 
computer classroom at the College of Education where 
students either took part in the experimental curriculum or 
the traditional curriculum depending on which section they 
chose to enroll in with no prior knowledge of the 
pedagogical or curricular approach, 


Data collection and analysis 

This study employed a quasi-experimental, posttest 
comparison design to measure the effect of a digital 
game-based, problem-based curriculum in a hybrid 
course on student achievement as compared with student 
achievement in the existing course curriculum. The posttest 
questions used the shared learning objectives for each 
section of the course as a means of developing common 
test items that would be given to students on both the 
pretest and final exam, which would act as the posttest. In 
the instance of the achievement questions, students were 
randomly assigned to a condition dependent upon the 
section of the course that they signed up to take with no 
prior knowledge of the research questions and with no 
undue influence by the researchers, One section engaged 
in the existing face-to-face curriculum and served as the 
comparison group. A second section, developed 
specifically as a hybrid course with six total face-to-face 
meetings combined with learning tasks and activities to be 
completed online using open-source and no-cost Web 2,0 
tools. This section acted as the treatment condition. 
Research Questions 

The following were our main research questions linked to 
overall goals of the course: 

Question 1 

Can the use of a game-driven, problem-based learning 
curriculum for post-secondary learners that leverages 
existing and developed distributed learning resources 
should improve the achievement of learners at a 
statistically significant level more than those learners in the 
existing drill and practice-based course? 

In order to address this question, students in the two 
conditions engaged with either the treatment or 
comparison curricula over the course of the semester. At 
three points during the semester (pretest, midterm and 
final), each group then completed exams based on the 
same learning objectives shared by each curriculum. For 
the purposes of the report, we only report the pre and 
posttest scores, as the midterm did not meet requirements 
for adequate validity as a testing instrument. 
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Question 2 

Can the use of a game-driven, problem-based learning 
curriculum for post-secondary learners that leverages 
existing and developed distributed learning resources 
improve the level of satisfaction expressed by learners 
more than those learners in the existing drill and practice- 
based course at a statistically significant level? 

This question was addressed by providing students with a 
survey of learner satisfaction with the overall course, the 
means of instruction, means of assessment, and learning 
activities similar to the means of course quality assessment 
that will be provided by the College of Education, While 
summative, this survey was more in-depth than the 
traditional five-question survey provided by the university. It 
asked specific questions about the delivery of instruction by 
the game and online systems, attitude toward instructor, 
attitude towards instructional style, self-report of 
instructional style, attitude towards individual learning 
components and activities, attitude towards peers, as well 
as general satisfaction with the course. 

Results 

The analysis of scores from first, Spring 2007, 
implementation of the experimental curriculum had 
mixed, but promising results on measures of retention, 
satisfaction, and achievement as shown in Table 1, 

The results indicate an 8.55% difference in the percent of 
students who dropped, failed, or withdrew between the 
comparison course and the treatment, which, while not 
significant, shows some improvement in raw numbers, 
though the groups are to small to reach the needed t- 
score. However, satisfaction with the redesigned course, as 
gauged by the college course evaluation which is the 



Comparison 

n=57 

Treatment 

n=32 

Differences 

Retention (% DFW) 

21.05% 

12.50% 

- 8.55% 

Drops 

1 

2 


Failures 

2 

0 


Withdrawals 

9 

2 


Satisfaction 

3.64 

4.2 

*alpha=.05, 
z(6)=6.86, p=1.64 

Achievement 

M = 78.83 

191=85.96 

*t=3.90,crit=l .67 


*statistically significant 

Table 1. Quantitative results for student retention, satisfaction, 
and achievement. 


main measure used by the university to measure student 
satisfaction with a course, was statistically significantly 
higher than in sections using the existing course design. 
Finally, student achievement, as measured by post-test in 
both groups, and compared using a two-sample t-test 
assuming unequal variances showed statistically 
significant improvement in the treatment group the 
comparison group, indicating that students retained more 
of the reified knowledge the authors expected them to 
learn than the old course curriculum. 

Limitations 

In the interests of space and scope, the report of the study 
here focuses on the quantitative findings while the 
qualitative findings may be found elsewhere so that they 
can be reported fully. This removes a major element from 
this particular report that provides additional context for the 
findings from this study. Further, instructor influence may 
have been a factor in the outcomes when comparing 
student outcomes on the achievement and satisfaction 
scores as they play a large role in student success. Had 
another instructor been chosen for the comparison course, 
it is possible no statistical significance would have been 
found. However, this is a major challenge in all research 
employing comparison and treatment groups. In order to 
address this in future research, the experimental course 
section will be compared against several different sections 
employing non-experimental methods. 

Sustainability and replication 

The course continues to be taught in different forms each 
semester. These differences stem from data collected 
each semester from students and faculty teaching the 
course. Upon reviewing the qualitative data collected to 
contextualize the quantitative findings, communication 
problems among students in their groups was paramount, 
indicating that this is a skill entering freshman substantially 
lack. This also had roots in the design of the course that 
required high levels of communication using digital tools as 
a means of completing work and asking questions. 
Problems of communication were further exacerbated by 
the size of some of the groups that ranged from three to 
five, depending on student choice. Finally, a lack of student 
experience with group communication and problem 
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solving that they brought with them to the course from high 
school and other undergraduate courses further 
complicated matters. 

Problem-based learning and communication 

In addition, while the problem-based learning component 
was challenging for some students, interviews conducted 
with students, as reported elsewhere (Warren, 2010; Warren 
& Dondlinger, 2009), they made clear linkages between 
problems they were solving in the course and those they 
would have to solve in the future. However, while several 
students reported disliking working in groups to complete 
tasks, they did recognize its necessity in their future careers 
and that excellent interpersonal communications skills are 
necessary for their future success. The next iteration of this 
course will undergo a full redesign in order to specifically 
target each of these communicative goals and allow for 
the evaluation of student and instructor success at 
reaching them. The authors will also redesign the problem- 
based aspect of the course and some game aspects so 
that students must complete the game in order to 
successfully complete the course, which we hope will result 
in more voluntary play and higher student satisfaction rates 
amongst students. 

Future research 

Future research will seek to confirm the findings of this 
particular study as well as to more deeply explore a large 
number of student experience by employing a number of 
different research tools including student web log (blog) 
reflections, bi-weekly interviews, and other forms of data 
collection to capture student experience as they work 
through the course. In addition, they will specifically focus 
on the mediating and aggravating roles of group 
communication, student self-regulated learning, problem- 
based learning methods, and the degree to which the 
game elements help or hinder learning. 

Conclusion 

While the instructional methods have yielded mixed results 
in the pilot research stemming from the use of problem- 
based learning and its accompanying reliance on 
students to self-organize and solve ill-structured problems, it 
has provided a wealth of data and results related to 
improving student experience with experimental and 


innovative instructional methods that push towards the 
edges of what our students are capable of as they enter 
college. Overall, the results of the research related to this 
course redesign leave the researchers hopeful that the 
hybrid course and use of an AltRG combined with PBL 
methods to frame the problem-based learning tasks and 
group interactions were responsible for the improved test 
scores. While the statistical findings of this study were mixed 
in terms of student retention, the increased post-test and 
ratings scores are promising and will be followed up with 
additional studies on future iterations of the course. 

Students were often not prepared by high school for the 
innovative curricula they should expect in college. 
Therefore, we must work more closely with professionals in K- 
12 settings and at state agencies to better prepare 
students and instructors for the critical thinking and creative 
tasks they should expect in college and beyond. 
Developing innovative curricula in high school that both 
targets acquisition of knowledge and skills while 
challenging students to solve ill-structured problems and 
be prepared for self-direction, ambiguity, and critical 
thinking will require that instructional designers and 
educators on both sides of the secondary/post-secondary 
line challenge the fundamental objectives of schooling, By 
doing so, we prepare our students for the future world of 
work in meaningful ways that allow them to be successful in 
the 21 st century Conceptual Age economy rather than be 
left behind because they are not prepared to adapt to the 
rapidly changing world around them. 
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